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ABSTRACT 

Jersey City State College (New Jersey) developed a 
local evaluation model to assess all salient aspects of the 
institution's basic skills program. The college's Academic 
Foundations Program was implemented to address the needs of urban 
educationally disadvantaged students. It includes faculty and staff 
development and research, freshman course placement based on student 
performance on a state-mandated placement test, and remedial courses 
in English, reading, and mathematics. Proficiency levels and 
remedial-course exit criteria are determined by student performance 
on course posttest measures. In developing the evaluation model six 
program objectives were formulated: (1) establishment of a cohesive 
structure for the basic skills program; (2) remedial 
rates; (3) first-year course outcomes; (4) skills 
skills course completers; (5) subsequent academic 
retention rates; (6) basic skills research conducted 
staff. Quality indicator standards were set based 
structure, characteristics of target and 
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peer institutions, and statewide policies with respect to basic 
skills. Qualitative and quantitative data were compiled. Results 
showed that five of the six objectives were met. The quality 
indicator standard was met in English and mathematics but not 
reading. Data showed that the basic skills program objectives 
met overall for the 1992-93 academic year. (Contains 14 
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THE IMPLEMENTATION OF A LOCAL MODEL 
TO ASSESS THE BASIC SKILLS PROGRAM 
AT A FOUR-YEAR PUBLIC COLLEGE 



A local evaluation model was developed to assess all salient aspects of the 
College basic skills program. In view of the needs of the freshman population, six 
program objectives were formulated. Objectives focused on the program structure, 
course placement indices, course outcomes, skill development, long-term 
performance and retention, and research. Quality indicator standards were set in 
view of the College organizational structure and the frame of reference provided by 
peer institutions. Qualitative and quantitative data were compiled. Results 
showed that five of the six objectives were met. The model may be adapted to 
assess programs at other postsecondary institutions. 



9 

ERJC 



4 



3 

A local model was developed to assess the basic skills program that is 
provided for freshman students at Jersey City State College. The usefulness of local 
models that reflect the student population and the programs offered at individual 
institutions has been cited by higher education researchers. Ewell (1985) has noted: 
"— outcomes assessment programs need to be carefully tailored to their 
institutional and curricular settings (p. 115)." Further, Pace (1990) has stated: "Each 
college should clarify — the clientele it is prepared to serve, and the achievements it 
expects of the students it admits (p.8)." Therefore, the Director of Institutional 
Research has developed a local model in view of the population served and the 
resources allocated at Jersey City State College, a four-year public New Jersey college. 

In so doing, it was important to gather information from a variety of sources 
to understand the needs of the entering freshmen and to facilitate the development 
of basic skills program objectives. Data indicated that, with respect to demographic 
and academic characteristics, the local students differed considerably from 
undergraduates at the other four-year New Jersey colleges (Lyons, 1992a, 1992b). 
Whereas the other state institutions enroll a predominantly traditional population, 
Jersey City State College is unique in serving an urban student group. In this 
regard, the majority of New Jersey college students are white, recent high school 
graduates who initially enroll full time in regular academic programs. On the other 
hand, the Jersey City State College undergraduate population is comprised of many 
adult, minority, financially and educationally disadvantaged students whose 
employment and college attendance patterns refkc* the life situations of adults who 
live in urban areas. 

This unique set of characteristics, indigenous to Jersey City State College 
students, is evident at the national level as well (California State University, 
Division of Analytical Studies, 1985, 1990; Davila, 1985a, 1985b; Murtha, Blumberg, 
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O'Dell, and Crook, 1989; Murtha, Protash, and Kaufman, 1983; Porter, 1989). In 
view of the fact that Jersey City State College students resemble more closely the 
national urban population, rather than students enrolled at other New Jersey 
colleges, the former student population may be viewed as an appropriate peer 
group. 

The identification of a student peer group is particularly critical, in view of 
New Jersey state mandates which require initial placement in basic skills courses for 
all freshmen who demonstrate a need for remediation. Consequently, in designing 
an assessment model at the local level, it is not feasible to provide a control group. 
In the absence of a control group, it is important to select an appropriate comparison 
group to facilitate the interpretation of outcome data. 
METHOD 

Objectives and Evaluation Design 

In order to formulate meaningful program objectives, it is essential to 
understand not only pertinent characteristics of the population served, but also the 
structure and components of the freshman basic skills program. In this regard, an 
Academic Foundations Program has been implemented to address the needs of the 
urban students who enroll at the College. Participating faculty and staff convene on 
an ongoing basis to review all pertinent curriculum issues, to determine program 
policies, and to design research projects. Freshman course placement is based on 
student performance on the state-mandated New Jersey College Basic Skills 
Placement Test. With respect to the College basic skills program, one remedial- 
level course is offered in English and in reading; two, in mathematics. These 
courses prepare students for college-level work in verbal and quantitative areas. 
Proficiency levels and remedial-course exit criteria are determined by student 
performance on course posttest measures. 
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In view of the needs of the basic skills population and the structure of the 
program as described above, the Director of Institutional Research formulated six 
program objectives. These short-term and long-term objectives were devised to 
assess the basic skills program structure; remedial-course placement rates; first-year 
course outcomes; skills developed by basic skills course completers; subsequent 
academic performance and retention rater-; and basic skills research conducted by 
College faculty and staff. 

To determine if these program objectives were met, quality indicator levels 
were determined. Standards in these areas were set in view of the program 
structure, characteristics of target and comparison groups, and statewide policies 
with respect to basic skills. Data in the above areas are reported as follows. 
RESULTS 
Program Outcomes 

Qualitative and quantitative data were compiled, as appropriate, to determine 
if the six program objectives were met. The findings are presented as follows. 
Objective 1: To establish a cohesive structure for the provision of basic skills 
courses and services 

Evaluation Design: The structure and functions of The Academic Foundations 
Program were delineated to determine if a viable mechanism was provided for 
disseminating information and reviewing policy. 

Outcome Data: The College has developed a formal structure which enables faculty 
and staff to address all critical basic skills issues. Central to this structure is the 
Academic Foundations Program. Overall planning is carried out by the Program's 
Coordinator and a Faculty and Staff Committee consisting of representatives of the 
departments and service units which participate in the Program. The full 
Committee convenes on a monthly basts and various subcommittees, periodically, 
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to review curriculum, testing, and evaluation policies, and to present 
recommendations to their respective departments. The above qualitative data 
show that the College has provided a viable mechanism for disseminating testing 
and course placement information and for reviewing policy issues on an ongoing 
basis. 

Objective: 2(a) To identify skills-deficient freshmen 2(b) To place skills-deficient 
freshmen in appropriate entry-level courses 

Evaluation Design: New Jersey College Basic Skills Placement Test data were 
analyzed to determine if (a) ninety percent of the full-time freshmen were tested, 
and appropriate proficiency standards, set and (b) ninety percent were placed in 
designated basic skills courses. 

Outcome Data: 2(a) Freshman test data were compiled and are reported in Tables 1 
and 2. 

TABLE 1. Identification and Testing of the 
Fall, 1992 Freshmen 

Freshman Group Full Time Part Time Total 

Placement Test Required 532 
Placement Test Administered 523 
Percentage of Freshmen Tested 98% 



149 681 
144 667 
97% 98% 
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TABLE 2. New Jersey Placement Test Cutoff Scores 

Established for Remedial Course Placement 



Subtest 


Cutoff 


Scores 3 


NJ State Standards 


Jersey City State College 


Reading Comprehension 


161 


162 


Essay 


7 


8 (local scores) 


Computation 


165 


165 


Elementary Algebra 


167 


173 



a Scores below which basic skills course enrollment is mandated 

As indicated in Table 1, placement tests were administered to 98% of the full- 
time, fall 1992 freshmen. Clearly, the College has exceeded the 90% quality indicator 
standard for full-time freshman testing. Further, as reported in Table 2, cutoff 
scores for placement in remedial-level courses at Jersey City State College equaled or 
exceeded the New Jersey standards. Consequently, local criteria for determining 
skill deficiencies were very much in line with statewide normative data in these 
areas. 

Outcome Data: 2(b) Remedial course placement data were tabulated, and results 
are presented in Table 3. 
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TABLE 3. First- Year Remedial Course Enrollments: 
Fall 1992 Freshmen 



Remedial Course 


Remediation Needed 3 


Course Enrollment 




Full Time 


Part Time 
N. %. 


Full Time 


Part Time 
N_ °A 


English: 

College Writing 


179 


34% 


69 


48% 


179 100% 


56 81% 


Math: 
Basic Math 


115 


. 22% 


95 


66% 


111 97% 


81 85% 


Math: 














Algebra for College 


230 


44% 


42 


29% 


226 98% 


36 86% 


Reading: 

Reading for College 


242 


46% 


89 


62% 


205 85% 


56 63% 



a Based on the placement test performance of fall, 1992 freshmen 



As shown above, a substantial proportion of full-time, fall 1992 freshmen 
demonstrated an initial need for remediation. Their first-year course enrollments 
indicate that 100% of those who required remediation in writing enrolled in the 
appropriate English course; 97%, in the appropriate basic math course; 98%, in 
algebra; 85%, in reading. It is apparent that the College has exceeded the 90% quality 
indicator standard for full-time freshman placement in remedial English and 
mathematics courses. With respect to reading, the quality indicator level of 85% was 
slightly lower than expected. 

Objective 3: To assess first-year course outcomes for fall, 1992 freshmen 
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Evaluation Design: Passing rates in first-and second-level freshman courses were 
compiled for the fall, 1992 cohort. Performance and completion rates were assessed 
in view of normative urban-peer data. 
Outcome Data: Results are summarized in Tables 4 and 5. 



TABLE 4. Remedial Course Passing Rates: 
First-Level Courses 



Remedial Course 


Full Time 


Part Time 




M 


% 




% 


English: College Writing 


110 


61% 


25 


45% 


Math: Basic Math 


93 


84% 


64 


79% 


Math: Algebra for College 


187 


83% 


29 


81% 


Reading: Reading for College 


179 


87% 


47 


84% 



TABLE 5. Course Passing Rates: Second-Level 
Courses in English and Math 



Course Full Time 


Part Time 


N 

Enrolled 


N 
Passed 


/o 

Passed 


N 

Enrolled 


N 
Passed 


0/ 
10 

Passed 


English: Fund. Comm I. 

1st Course: 

College Writing 93 
Fund. Comm. I 318 


81 
219 


87% 
69% 


23 
70 


18 
55 


78% 
79% 


Math: Algebra for College 

1st Course: 

Basic Math 83 
Algebra for College 226 


52 
187 


63% 
83% 


38 
36 


25 
29 


66% 
81% 
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Passing rates in first-level remedial courses are presented in Table 4. As 
shown, more than four-fifths of the full-time freshmen passed their initial courses 
in mathematics and reading; more than three-fifths, in English. 

Follow-up data in second-level courses are reported in Table 5 for comparison 
groups in English and math. With respect to English, passing rates in 
Fundamentals of Communication I, the freshman-level writing course, were greater 
for full-time students who initially enrolled in and passed the remedial-level course 
(87%) than for freshmen who did not need remediation in writing (69%). With 
respect to mathematics, as shown in Table 5, passing rates in Algebra for College, the 
second-level remedial math course, were favorable but were lower for full-time 
freshmen who initially required remediation in computation (63%) than for those 
who demonstrated no computation deficiencies (83%). 

As reported, the vast majority of freshmen passed the first-and second-level 
courses in English, -math, and reading during their first year at the College. These 
findings are especially meaningful in view of the performance levels of the national 
urban student population. In this regard, many higher education researchers have 
described the challenges confronted by urban students in their academic transition 
to college (Richardson and Bender, 1985; Terenzini, 1993; Tinto, 1987). These 
challenges often have a negative impact on initial academic performance and 
course completion rates. Particularly in view of these trends, local course outcomes 
may be viewed as favorable, and the quality indicator levels attained by the majority 
of freshmen, most acceptable. 

Objective 4: To assess growth in the respective areas for fall, 1992 basic skills 
students 

Evaluation Design: Departmental and New Jersey state posttest data were analyzed 
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to determine if posttests were administered to 95% of the 1992-93 course completers. 
Additionally, pre-and posttest data in reading and math were analyzed to determine 
if students showed significant growth in the respective skill areas. 
Outcome Data: Two posttest measures in reading and mathematics, i.e., the New 
Jersey and the departmental instruments, were administered. Both measures were 
utilized to assess proficiency in mathematics; only the departmental test, in reading. 
Mean scores on the pre-and posttest measures were obtained, and t-tests were 
computed to assess growth in the various skill areas. Results are reported in Tables 
6 and 7. 

TABLE 6: Analysis of New Jersey College Basic 

Skills Test Data in Reading and Mathematics 
For 1992-93 Course Completers 



Pretest Scores Posttest Scores 

Subtest N Mean SD Mean SD t 



Reading Comprehension 226 150.01 7.89 161.27 8.09 19.44*** 
Computation 157 157.58 5.38 170.26 5.04 27.09*** 

Elementary Algebra 216 163.24 5.75 176.54 5.66 29.47*** 



*** p <.001 
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TABLE 7. Analysis of Departmental Test Data in 
Reading and Mathematics for 1992-93 
Course Completers 



Subtest 




Pretest Scores 


Posttest Scores 




N 


Mean 


SD 


Mean 


SD 


t 


Reading Comprehension 


226 


25.25 


3.77 


32.77 


3.13 


25.44*** 


Computation 


157 


8.04 


5.12 


21.94 


5.44 


26.75*** 


Elementary Algebra 


216 


9.16 


4.52 


22.76 


3.77 


34.91*** 



p <.001 



The number of students tested represents 100% of the students who 
completed Reading for College; Basic Math; and Algebra for College during the 
1992-93 academic year. Clearly, the College has exceeded the 95% quality indicator 
standard for posttesting course completers. It should be noted that, a two-hour 
Minimum Competency Essay Test, scored by the English faculty Grading 
Committee, was administered to all students who completed the remedial-level 
English course. The essay was devised to assess mastery of freshman-level writing 
skills. However, in view of the lack of comparability between placement and 
posttest instruments, it was not possible to analyze the essay data on a pre-and 
posttest basis. 

As shown in Tables 6 and 7, students enrolled in Reading for College showed 
significant growth in Reading Comprehension on the New Jersey Basic Skills Test, 
as well as the departmental test Students enrolled in Basic Math made significant 
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gains in Computation on the New Jersey and the departmental tests. Finally, 
students who completed Algebra for College showed significant growth in 
Elementary Algebra on the New Jersey as well as the departmental tests. The 
significant gains demonstrated by course completers in reading and mathematics 
provide ample evidence to show that objective four has been met. 
Objective 5: To assess outcomes for freshman cohorts in subsequent years 
Evaluation Design: The academic performance and retention rates of previous 
freshman cohorts were analyzed by initial basic skills proficiency levels. Long-term 
performance and retention were assessed in view of urban-peer data. 
Outcome Data: To understand student retention patterns over time, full-time 
freshmen who initially enrolled at the College in the fall 1989, the fall 1990, and the 
fall 1991 have been identified. Based on their New Jersey College Basic Skills 
Placement Test performance, the students were grouped into four categories, i.e., 
those who initially demonstrated proficiency in all areas (reading, writing, and 
computation); and those who demonstrated a need for skill development in one 
area; two areas; or three areas. Enrollment patterns and performance levels over 
time for students in each of the above categories are presented in Tables 8 and 9. 
Chi-square analyses were performed to determine if there were significant 
differences over time among students at various skill levels. 
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TABLE 8; 


Long-Term Retention Rates For 










Freshmen at Various Skill Levels . 










Skill Level 








Retention Year 


Proficient 


One Skill 


Two Skills 


Three Skills 


i ocai 




Two Years 












221 


Retained 


60 


iUi 


12U 


36 


317 




Not Retained 


44 


57 


68 


28 


ly/ 




Total Cohort 


104 


158 


188 


64 






Three Years 












0.61 


Retained 


49 


81 


98 


31 


259 




Not Retained 


55 


77 


90 


33 


255 




Total Cohort 


104 


158 


188 


64 


514 




Four Years 












2.37 


Retained 


45 


74 


90 


24 


233 




iNot Ketaineci 




84 


98 


40 


281 




Total Cohort 


104 


158 


188 


64 


514 






TABLE 9: 


Long-Term Performance Levels For 










Freshmen at Various Skill Levels 










Skill Level 








Performance Year 


Proficient 


One Skill 


Two Skills 


Three Skills 


Total 




Two Years 










17.35*** 


GPA_> 2.0 


55 


91 


92 


24 


262 




GPA < 2.0 


5 


10 


28 


12 


55 




Total Enrolled 


60 


101 


120 


36 


317 




Three Years 










28.21*** 


GPA.^2.0 


47 


75 


86 


18 


226 




GPA < 2.0 


2 


6 


12 


13 


33 




Total Enrolled 


49 


81 


98 


31 


259 




Four Years 










3.36 


GPA > 2.0 


42 


68 


88 


21 


219 




GPA < 2.0 


3 


6 


2 


3 


14 




Total Enrolled 


45 


74 


90 


24 


233 





p < .001 
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Data reported in Table 8 for three freshman cohorts show that attrition was 
greatest within the first two years of initial enrollment at the College, and continued 
at a slower rate in subsequent years. Overall, attrition has been slightly higher for 
freshmen with three skill deficiencies than for other freshman groups. However, 
results of the chi-square analyses showed that differences were not significant. 

Data to show long-term academic performance for the various freshman 
groups have been presented in Table 9. With respect to performance by skill levels, 
significant differences were found after two and three years according to the 
students' initial skill proficiencies in verbal and quantitative areas. 

Overall retention patterns at Jersey City State College were similar to those at 
other urban public institutions. As noted by Porter (1989): "The greatest single 
point of loss in persistence is the first year (p. 5)." Further, the role of entry-level 
skills in the retention patterns of urban students has been cited by various 
researchers. In a recent California State University publication (1990), it was stated: 
"The persistence rate for regular, admits after five years (was) 54.8 percent — for 
special admissions was 33.2 percent — (p. 9)." Further, Murtha et al. (1989), in 
discussing trends at The City University of New York, have noted: "The number of 
skills tests passed at entry also directly correlates with graduation and persistence. -- 
University-wide, only a little more than one-fourth of those who failed all three 
skills tests persisted (p. 17)." 

As indicated by higher education researchers, outcomes for underprepared 
freshmen have been less favorable than for academically proficient students. These 
national trends were evident to some extent at the local level, as well. However, 
differences over time between remedial and regular students at Jersey City State 
College were considerably less pronounced than at other urban institutions. 
Consequently, the long-term retention and performance levels at Jersey City State 
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College compared favorably with, and, in fact, often exceeded those at many urban 
peer institutions throughout the country. These findings provide evidence to 
show that objective five has been met. 

Objective 6: To enhance the College's understanding of student needs, basic skills 
research has been conducted by College faculty and staff. 

Evaluation Design: Research projects conducted by faculty and staff have been 
described; projects completed and presented to professional constituencies within 
the previous two years have been reported. 

Outcome Data: Research has included projects conducted by the Mathematics Area. 
Coordinator in collaboration with the Electronic Learning Laboratory 
Administrative Assistant; the Chairperson of the Reading Department; and the 
Coordinator of Freshman Testing. These research papers were entitled, 
respectively: The use of computer assisted instruction in developmental math: 
Effects of a precollege reading co u rse on the academic self-esteem of urban college 
Studen ts: and Tested freshmen wh o did not enroll: Pertinent characteristics and 
subsequent educational rigrisirmft, Fall 1991. Two of these studies were presented at 
annual meetings of the New Jersey Association of Developmental Education; one, at 
the annual meeting of the College Reading Association. The above research has 
fostered a greater understanding of the academic and non-academic needs of the 
freshman students and has provided ample evidence to show that objective six has 
been met. 
DISCUSSION 

As reported in the previous section of this paper, evidence has been provided 
to show that five of the six program objectives have been met. With respect to basic 
skills course placements for full-time freshmen, the 90% quality indicator standard 
was met in English and mathematics, but not in reading. It may be appropriate 
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to assess the 85% course placement index in reading in view of the lack of a reading 
course requirement for graduation. In light of these findings, College faculty and 
staff are now reviewing this policy so that enrollment rates in the appropriate 
reading course may be enhanced. 

With the exception of the reading course component of objective two, data 
have shown that the basic skills program objectives were met for the 1992-93 
academic year. In this regard, outcomes were favorable with respect to the basic 
skills program structure, freshman course passing rates and skill development, 
subsequent college performance and retention, and pertinent faculty and staff 
research projects. 

Overall, in developing an evaluation design to assess the effectiveness of the 
basic skills program at a public urban college, it has been necessary to address 
methodological as well as practical issues. As noted by higher education researchers, 
in conducting an evaluation at the institutional level, it is important to consider all 
relevant population and program factors. Therefore, the Director of Institutional 
Research consulted a variety of data sources to understand the salient characteristics 
of the population served; to identify pertinent comparison groups; and to gather 
information on curricular and policy issues that may impact on program 
implementation. As a result, it has been possible to formulate program objectives 
that are consonant with the needs of the population and to provide a meaningful 
frame of reference for assessing program outcomes. 
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